Clarifying Survey Questions When Respondents Don't Know They Need Clarification

Enabling respondents to obtain clarification about the intended meaning of survey questions can dramatically improve response accuracy. However, respondents often do not recognize that their conceptions of terms in a question can differ from the question author's conception. As a result they do not request clarification on many occasions when it could improve their understanding and, thus, the accuracy of their responses. Similarly, survey interviewers often do not recognize that conceptions are misaligned and so do not volunteer clarification when it may be most helpful. We have been exploring two approaches to the design of web-based questionnaires that could increase the number of occasions when helpful clarification is provided. One approach is to increase respondents’ sensitivity to the possibility of conceptual misalignment by rewording questions so that they include part of the definition of the key concept(s). We found that under some circumstances this led to increased requests for the full definition. Another approach is to build respondent models into the questionnaires so that the survey system can determine when respondents are confused and volunteer clarification. Such models can vary in their specificity from generic respondents to groups of respondents (stereotypic models) to models of individual respondents. We discuss ongoing work on respondent modeling.